{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a data-flickr-embed=\"true\"  href=\"https://www.flickr.com/photos/157237655@N08/46489714642/in/datetaken-public/\" title=\"YOLO model training in progress\"><img src=\"https://farm8.staticflickr.com/7840/46489714642_d69661a409_b.jpg\" width=\"1024\" height=\"797\" alt=\"YOLO model training in progress\"></a><script async src=\"//embedr.flickr.com/assets/client-code.js\" charset=\"utf-8\"></script>\n",
    "\n",
    "This is the fifth blog post of [Object Detection with YOLO blog series](https://fairyonice.github.io/tag/object-detection-using-yolov2-on-pascal-voc2012-series.html). This blog finally train the model using the scripts that are developed in the [previous blog posts](https://fairyonice.github.io/tag/object-detection-using-yolov2-on-pascal-voc2012-series.html). \n",
    "I will use PASCAL VOC2012 data. \n",
    "This blog assumes that the readers have read the previous blog posts - [Part 1](https://fairyonice.github.io/Part_1_Object_Detection_with_Yolo_for_VOC_2014_data_anchor_box_clustering.html), [Part 2](https://fairyonice.github.io/Part%202_Object_Detection_with_Yolo_using_VOC_2014_data_input_and_output_encoding.html), [Part 3](https://fairyonice.github.io/Part_3_Object_Detection_with_Yolo_using_VOC_2012_data_model.html), [Part 4](https://fairyonice.github.io/Part_4_Object_Detection_with_Yolo_using_VOC_2012_data_loss.html).\n",
    "\n",
    "## Andrew Ng's YOLO lecture\n",
    "- [Neural Networks - Bounding Box Predictions](https://www.youtube.com/watch?v=gKreZOUi-O0&t=0s&index=7&list=PL_IHmaMAvkVxdDOBRg2CbcJBq9SY7ZUvs)\n",
    "- [C4W3L06 Intersection Over Union](https://www.youtube.com/watch?v=ANIzQ5G-XPE&t=7s)\n",
    "- [C4W3L07 Nonmax Suppression](https://www.youtube.com/watch?v=VAo84c1hQX8&t=192s)\n",
    "- [C4W3L08 Anchor Boxes](https://www.youtube.com/watch?v=RTlwl2bv0Tg&t=28s)\n",
    "- [C4W3L09 YOLO Algorithm](https://www.youtube.com/watch?v=9s_FpMpdYW8&t=34s)\n",
    "\n",
    "## Reference\n",
    "- [You Only Look Once:Unified, Real-Time Object Detection](https://arxiv.org/pdf/1506.02640.pdf) \n",
    "\n",
    "- [YOLO9000:Better, Faster, Stronger](https://arxiv.org/pdf/1612.08242.pdf)\n",
    " \n",
    "- [experiencor/keras-yolo2](https://github.com/experiencor/keras-yolo2)\n",
    "\n",
    "## Reference in my blog\n",
    "- [Part 1 Object Detection using YOLOv2 on Pascal VOC2012 - anchor box clustering](https://fairyonice.github.io/Part_1_Object_Detection_with_Yolo_for_VOC_2014_data_anchor_box_clustering.html)\n",
    "- [Part 2 Object Detection using YOLOv2 on Pascal VOC2012 - input and output encoding](https://fairyonice.github.io/Part%202_Object_Detection_with_Yolo_using_VOC_2014_data_input_and_output_encoding.html)\n",
    "- [Part 3 Object Detection using YOLOv2 on Pascal VOC2012 - model](https://fairyonice.github.io/Part_3_Object_Detection_with_Yolo_using_VOC_2012_data_model.html)\n",
    "- [Part 4 Object Detection using YOLOv2 on Pascal VOC2012 - loss](https://fairyonice.github.io/Part_4_Object_Detection_with_Yolo_using_VOC_2012_data_loss.html)\n",
    "- [Part 5 Object Detection using YOLOv2 on Pascal VOC2012 - training](https://fairyonice.github.io/Part_5_Object_Detection_with_Yolo_using_VOC_2012_data_training.html)\n",
    "- [Part 6 Object Detection using YOLOv2 on Pascal VOC 2012 data - inference on image](https://fairyonice.github.io/Part_6_Object_Detection_with_Yolo_using_VOC_2012_data_inference_image.html)\n",
    "- [Part 7 Object Detection using YOLOv2 on Pascal VOC 2012 data - inference on video](https://fairyonice.github.io/Part_7_Object_Detection_with_Yolo_using_VOC_2012_data_inference_video.html)\n",
    "\n",
    "## My GitHub repository \n",
    "This repository contains all the ipython notebooks in this blog series and the funcitons (See backend.py). \n",
    "- [FairyOnIce/ObjectDetectionYolo](https://github.com/FairyOnIce/ObjectDetectionYolo)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/yumikondo/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.6.3 |Anaconda, Inc.| (default, Oct  6 2017, 12:04:38) \n",
      "[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os, sys\n",
    "import tensorflow as tf\n",
    "print(sys.version)\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define anchor box\n",
    "<code>ANCHORS</code> defines the number of anchor boxes and the shape of each anchor box.\n",
    "The choice of the anchor box specialization is already discussed in [Part 1 Object Detection using YOLOv2 on Pascal VOC2012 - anchor box clustering](https://fairyonice.github.io/Part_1_Object_Detection_with_Yolo_for_VOC_2014_data_anchor_box_clustering.html). \n",
    "\n",
    "Based on the K-means analysis in the previous blog post, I will select 4 anchor boxes of following width and height. The width and heights are rescaled in the grid cell scale (Assuming that the number of grid size is 13 by 13.) See [Part 2 Object Detection using YOLOv2 on Pascal VOC2012 - input and output encoding](https://fairyonice.github.io/Part%202_Object_Detection_with_Yolo_using_VOC_2014_data_input_and_output_encoding.html) to learn how I rescal the anchor box shapes into the grid cell scale.\n",
    "\n",
    "Here I choose 4 anchor boxes. With 13 by 13 grids, every frame gets 4 x 13 x 13 = 676 bouding box predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ANCHORS = np.array([1.07709888,  1.78171903,  # anchor box 1, width , height\n",
    "                    2.71054693,  5.12469308,  # anchor box 2, width,  height\n",
    "                   10.47181473, 10.09646365,  # anchor box 3, width,  height\n",
    "                    5.48531347,  8.11011331]) # anchor box 4, width,  height"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define Label vector containing 20 object classe names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "LABELS = ['aeroplane',  'bicycle', 'bird',  'boat',      'bottle', \n",
    "          'bus',        'car',      'cat',  'chair',     'cow',\n",
    "          'diningtable','dog',    'horse',  'motorbike', 'person',\n",
    "          'pottedplant','sheep',  'sofa',   'train',   'tvmonitor']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Read images and annotations into memory\n",
    "Use the pre-processing code for parsing annotation at [experiencor/keras-yolo2](https://github.com/experiencor/keras-yolo2).\n",
    "This <code>parse_annoation</code> function is already used in [Part 1 Object Detection using YOLOv2 on Pascal VOC2012 - anchor box clustering](https://fairyonice.github.io/Part_1_Object_Detection_with_Yolo_for_VOC_2014_data_anchor_box_clustering.html) and saved in my python script. \n",
    "This script can be downloaded at [my Github repository, FairyOnIce/ObjectDetectionYolo/backend](https://github.com/FairyOnIce/ObjectDetectionYolo/blob/master/backend.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "N train = 17125\n"
     ]
    }
   ],
   "source": [
    "### The location where the VOC2012 data is saved.\n",
    "train_image_folder = \"../ObjectDetectionRCNN/VOCdevkit/VOC2012/JPEGImages/\"\n",
    "train_annot_folder = \"../ObjectDetectionRCNN/VOCdevkit/VOC2012/Annotations/\"\n",
    "\n",
    "np.random.seed(1)\n",
    "from backend import parse_annotation\n",
    "train_image, seen_train_labels = parse_annotation(train_annot_folder,\n",
    "                                                  train_image_folder, \n",
    "                                                  labels=LABELS)\n",
    "print(\"N train = {}\".format(len(train_image)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Instantiate batch generator object\n",
    "<code>SimpleBatchGenerator</code> is discussed and used in \n",
    "[Part 2 Object Detection using YOLOv2 on Pascal VOC2012 - input and output encoding](https://fairyonice.github.io/Part%202_Object_Detection_with_Yolo_using_VOC_2014_data_input_and_output_encoding.html).\n",
    "This script can be downloaded at [my Github repository, FairyOnIce/ObjectDetectionYolo/backend](https://github.com/FairyOnIce/ObjectDetectionYolo/blob/master/backend.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from backend import SimpleBatchGenerator\n",
    "\n",
    "BATCH_SIZE        = 200\n",
    "IMAGE_H, IMAGE_W  = 416, 416\n",
    "GRID_H,  GRID_W   = 13 , 13\n",
    "TRUE_BOX_BUFFER   = 50\n",
    "BOX               = int(len(ANCHORS)/2)\n",
    "\n",
    "generator_config = {\n",
    "    'IMAGE_H'         : IMAGE_H, \n",
    "    'IMAGE_W'         : IMAGE_W,\n",
    "    'GRID_H'          : GRID_H,  \n",
    "    'GRID_W'          : GRID_W,\n",
    "    'LABELS'          : LABELS,\n",
    "    'ANCHORS'         : ANCHORS,\n",
    "    'BATCH_SIZE'      : BATCH_SIZE,\n",
    "    'TRUE_BOX_BUFFER' : TRUE_BOX_BUFFER,\n",
    "}\n",
    "\n",
    "\n",
    "def normalize(image):\n",
    "    return image / 255.\n",
    "train_batch_generator = SimpleBatchGenerator(train_image, generator_config,\n",
    "                                             norm=normalize, shuffle=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define model\n",
    "We define a YOLO model.\n",
    "The model defenition function is already discussed in [Part 3 Object Detection using YOLOv2 on Pascal VOC2012 - model](https://fairyonice.github.io/Part_3_Object_Detection_with_Yolo_using_VOC_2012_data_model.html) and all the codes are available at [my Github](https://github.com/FairyOnIce/ObjectDetectionYolo/blob/master/backend.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "input_image (InputLayer)        (None, 416, 416, 3)  0                                            \n",
      "__________________________________________________________________________________________________\n",
      "conv_1 (Conv2D)                 (None, 416, 416, 32) 864         input_image[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "norm_1 (BatchNormalization)     (None, 416, 416, 32) 128         conv_1[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_1 (LeakyReLU)       (None, 416, 416, 32) 0           norm_1[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "maxpool1_416to208 (MaxPooling2D (None, 208, 208, 32) 0           leaky_re_lu_1[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "conv_2 (Conv2D)                 (None, 208, 208, 64) 18432       maxpool1_416to208[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "norm_2 (BatchNormalization)     (None, 208, 208, 64) 256         conv_2[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_2 (LeakyReLU)       (None, 208, 208, 64) 0           norm_2[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "maxpool1_208to104 (MaxPooling2D (None, 104, 104, 64) 0           leaky_re_lu_2[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "conv_3 (Conv2D)                 (None, 104, 104, 128 73728       maxpool1_208to104[0][0]          \n",
      "__________________________________________________________________________________________________\n",
      "norm_3 (BatchNormalization)     (None, 104, 104, 128 512         conv_3[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_3 (LeakyReLU)       (None, 104, 104, 128 0           norm_3[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "conv_4 (Conv2D)                 (None, 104, 104, 64) 8192        leaky_re_lu_3[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "norm_4 (BatchNormalization)     (None, 104, 104, 64) 256         conv_4[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_4 (LeakyReLU)       (None, 104, 104, 64) 0           norm_4[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "conv_5 (Conv2D)                 (None, 104, 104, 128 73728       leaky_re_lu_4[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "norm_5 (BatchNormalization)     (None, 104, 104, 128 512         conv_5[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_5 (LeakyReLU)       (None, 104, 104, 128 0           norm_5[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "maxpool1_104to52 (MaxPooling2D) (None, 52, 52, 128)  0           leaky_re_lu_5[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "conv_6 (Conv2D)                 (None, 52, 52, 256)  294912      maxpool1_104to52[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "norm_6 (BatchNormalization)     (None, 52, 52, 256)  1024        conv_6[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_6 (LeakyReLU)       (None, 52, 52, 256)  0           norm_6[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "conv_7 (Conv2D)                 (None, 52, 52, 128)  32768       leaky_re_lu_6[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "norm_7 (BatchNormalization)     (None, 52, 52, 128)  512         conv_7[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_7 (LeakyReLU)       (None, 52, 52, 128)  0           norm_7[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "conv_8 (Conv2D)                 (None, 52, 52, 256)  294912      leaky_re_lu_7[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "norm_8 (BatchNormalization)     (None, 52, 52, 256)  1024        conv_8[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_8 (LeakyReLU)       (None, 52, 52, 256)  0           norm_8[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "maxpool1_52to26 (MaxPooling2D)  (None, 26, 26, 256)  0           leaky_re_lu_8[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "conv_9 (Conv2D)                 (None, 26, 26, 512)  1179648     maxpool1_52to26[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "norm_9 (BatchNormalization)     (None, 26, 26, 512)  2048        conv_9[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_9 (LeakyReLU)       (None, 26, 26, 512)  0           norm_9[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "conv_10 (Conv2D)                (None, 26, 26, 256)  131072      leaky_re_lu_9[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "norm_10 (BatchNormalization)    (None, 26, 26, 256)  1024        conv_10[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_10 (LeakyReLU)      (None, 26, 26, 256)  0           norm_10[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_11 (Conv2D)                (None, 26, 26, 512)  1179648     leaky_re_lu_10[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "norm_11 (BatchNormalization)    (None, 26, 26, 512)  2048        conv_11[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_11 (LeakyReLU)      (None, 26, 26, 512)  0           norm_11[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_12 (Conv2D)                (None, 26, 26, 256)  131072      leaky_re_lu_11[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "norm_12 (BatchNormalization)    (None, 26, 26, 256)  1024        conv_12[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_12 (LeakyReLU)      (None, 26, 26, 256)  0           norm_12[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_13 (Conv2D)                (None, 26, 26, 512)  1179648     leaky_re_lu_12[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "norm_13 (BatchNormalization)    (None, 26, 26, 512)  2048        conv_13[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_13 (LeakyReLU)      (None, 26, 26, 512)  0           norm_13[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "maxpool1_26to13 (MaxPooling2D)  (None, 13, 13, 512)  0           leaky_re_lu_13[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "conv_14 (Conv2D)                (None, 13, 13, 1024) 4718592     maxpool1_26to13[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "norm_14 (BatchNormalization)    (None, 13, 13, 1024) 4096        conv_14[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_14 (LeakyReLU)      (None, 13, 13, 1024) 0           norm_14[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_15 (Conv2D)                (None, 13, 13, 512)  524288      leaky_re_lu_14[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "norm_15 (BatchNormalization)    (None, 13, 13, 512)  2048        conv_15[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_15 (LeakyReLU)      (None, 13, 13, 512)  0           norm_15[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_16 (Conv2D)                (None, 13, 13, 1024) 4718592     leaky_re_lu_15[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "norm_16 (BatchNormalization)    (None, 13, 13, 1024) 4096        conv_16[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_16 (LeakyReLU)      (None, 13, 13, 1024) 0           norm_16[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_17 (Conv2D)                (None, 13, 13, 512)  524288      leaky_re_lu_16[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "norm_17 (BatchNormalization)    (None, 13, 13, 512)  2048        conv_17[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_17 (LeakyReLU)      (None, 13, 13, 512)  0           norm_17[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_18 (Conv2D)                (None, 13, 13, 1024) 4718592     leaky_re_lu_17[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "norm_18 (BatchNormalization)    (None, 13, 13, 1024) 4096        conv_18[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_18 (LeakyReLU)      (None, 13, 13, 1024) 0           norm_18[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_19 (Conv2D)                (None, 13, 13, 1024) 9437184     leaky_re_lu_18[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "norm_19 (BatchNormalization)    (None, 13, 13, 1024) 4096        conv_19[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_21 (Conv2D)                (None, 26, 26, 64)   32768       leaky_re_lu_13[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_19 (LeakyReLU)      (None, 13, 13, 1024) 0           norm_19[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "norm_21 (BatchNormalization)    (None, 26, 26, 64)   256         conv_21[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_20 (Conv2D)                (None, 13, 13, 1024) 9437184     leaky_re_lu_19[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_21 (LeakyReLU)      (None, 26, 26, 64)   0           norm_21[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "norm_20 (BatchNormalization)    (None, 13, 13, 1024) 4096        conv_20[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "lambda_1 (Lambda)               (None, 13, 13, 256)  0           leaky_re_lu_21[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_20 (LeakyReLU)      (None, 13, 13, 1024) 0           norm_20[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_1 (Concatenate)     (None, 13, 13, 1280) 0           lambda_1[0][0]                   \n",
      "                                                                 leaky_re_lu_20[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "conv_22 (Conv2D)                (None, 13, 13, 1024) 11796480    concatenate_1[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "norm_22 (BatchNormalization)    (None, 13, 13, 1024) 4096        conv_22[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_re_lu_22 (LeakyReLU)      (None, 13, 13, 1024) 0           norm_22[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_23 (Conv2D)                (None, 13, 13, 100)  102500      leaky_re_lu_22[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "final_output (Reshape)          (None, 13, 13, 4, 25 0           conv_23[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "input_hack (InputLayer)         (None, 1, 1, 1, 50,  0                                            \n",
      "__________________________________________________________________________________________________\n",
      "hack_layer (Lambda)             (None, 13, 13, 4, 25 0           final_output[0][0]               \n",
      "                                                                 input_hack[0][0]                 \n",
      "==================================================================================================\n",
      "Total params: 50,650,436\n",
      "Trainable params: 102,500\n",
      "Non-trainable params: 50,547,936\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from backend import define_YOLOv2, set_pretrained_weight, initialize_weight\n",
    "CLASS             = len(LABELS)\n",
    "model, true_boxes = define_YOLOv2(IMAGE_H,IMAGE_W,GRID_H,GRID_W,TRUE_BOX_BUFFER,BOX,CLASS, \n",
    "                                  trainable=False)\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize the weights\n",
    "The initialization of weights are already discussed in [Part 3 Object Detection using YOLOv2 on Pascal VOC2012 - model](https://fairyonice.github.io/Part_3_Object_Detection_with_Yolo_using_VOC_2012_data_model.html). \n",
    "All the codes from [Part 3](https://fairyonice.github.io/Part_3_Object_Detection_with_Yolo_using_VOC_2012_data_model.html) are stored at [my Github](https://github.com/FairyOnIce/ObjectDetectionYolo/blob/master/backend.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "path_to_weight = \"./yolov2.weights\"\n",
    "nb_conv        = 22\n",
    "model          = set_pretrained_weight(model,nb_conv, path_to_weight)\n",
    "layer          = model.layers[-4] # the last convolutional layer\n",
    "initialize_weight(layer,sd=1/(GRID_H*GRID_W))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loss function\n",
    "We already discussed the loss function of YOLOv2 implemented by [experiencor/keras-yolo2](https://github.com/experiencor/keras-yolo2) in [Part 4 Object Detection using YOLOv2 on Pascal VOC2012 - loss](https://fairyonice.github.io/Part_4_Object_Detection_with_Yolo_using_VOC_2012_data_loss.html).\n",
    "I modified the codes and the codes are available at [my Github](https://github.com/FairyOnIce/ObjectDetectionYolo/blob/master/backend.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function custom_loss_core in module backend:\n",
      "\n",
      "custom_loss_core(y_true, y_pred, true_boxes, GRID_W, GRID_H, BATCH_SIZE, ANCHORS, LAMBDA_COORD, LAMBDA_CLASS, LAMBDA_NO_OBJECT, LAMBDA_OBJECT)\n",
      "    y_true : (N batch, N grid h, N grid w, N anchor, 4 + 1 + N classes)\n",
      "    y_true[irow, i_gridh, i_gridw, i_anchor, :4] = center_x, center_y, w, h\n",
      "    \n",
      "        center_x : The x coordinate center of the bounding box.\n",
      "                   Rescaled to range between 0 and N gird  w (e.g., ranging between [0,13)\n",
      "        center_y : The y coordinate center of the bounding box.\n",
      "                   Rescaled to range between 0 and N gird  h (e.g., ranging between [0,13)\n",
      "        w        : The width of the bounding box.\n",
      "                   Rescaled to range between 0 and N gird  w (e.g., ranging between [0,13)\n",
      "        h        : The height of the bounding box.\n",
      "                   Rescaled to range between 0 and N gird  h (e.g., ranging between [0,13)\n",
      "                   \n",
      "    y_true[irow, i_gridh, i_gridw, i_anchor, 4] = ground truth confidence\n",
      "        \n",
      "        ground truth confidence is 1 if object exists in this (anchor box, gird cell) pair\n",
      "    \n",
      "    y_true[irow, i_gridh, i_gridw, i_anchor, 5 + iclass] = 1 if the object is in category <iclass> else 0\n",
      "    \n",
      "    =====================================================\n",
      "    tensor that connect to the YOLO model's hack input \n",
      "    =====================================================    \n",
      "    \n",
      "    true_boxes    \n",
      "    \n",
      "    =========================================\n",
      "    training parameters specification example \n",
      "    =========================================\n",
      "    GRID_W             = 13\n",
      "    GRID_H             = 13\n",
      "    BATCH_SIZE         = 34\n",
      "    ANCHORS = np.array([1.07709888,  1.78171903,  # anchor box 1, width , height\n",
      "                        2.71054693,  5.12469308,  # anchor box 2, width,  height\n",
      "                       10.47181473, 10.09646365,  # anchor box 3, width,  height\n",
      "                        5.48531347,  8.11011331]) # anchor box 4, width,  height\n",
      "    LAMBDA_NO_OBJECT = 1.0\n",
      "    LAMBDA_OBJECT    = 5.0\n",
      "    LAMBDA_COORD     = 1.0\n",
      "    LAMBDA_CLASS     = 1.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from backend import custom_loss_core \n",
    "help(custom_loss_core)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that this custom function <code>custom_loss_core</code> depends not only on <code>y_true</code> and <code>y_pred</code> but also the various hayperparameters.\n",
    "Unfortunately, Keras's loss function API does not accept any parameters except <code>y_true</code> and <code>y_pred</code>. Therefore, these hyperparameters need to be defined globaly. \n",
    "To do this, I will define a wrapper function <code>custom_loss</code>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "GRID_W             = 13\n",
    "GRID_H             = 13\n",
    "BATCH_SIZE         = 34\n",
    "LAMBDA_NO_OBJECT = 1.0\n",
    "LAMBDA_OBJECT    = 5.0\n",
    "LAMBDA_COORD     = 1.0\n",
    "LAMBDA_CLASS     = 1.0\n",
    "    \n",
    "def custom_loss(y_true, y_pred):\n",
    "    return(custom_loss_core(\n",
    "                     y_true,\n",
    "                     y_pred,\n",
    "                     true_boxes,\n",
    "                     GRID_W,\n",
    "                     GRID_H,\n",
    "                     BATCH_SIZE,\n",
    "                     ANCHORS,\n",
    "                     LAMBDA_COORD,\n",
    "                     LAMBDA_CLASS,\n",
    "                     LAMBDA_NO_OBJECT, \n",
    "                     LAMBDA_OBJECT))\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training starts here! \n",
    "Finally, we start the training here.\n",
    "We only train the final 23rd layer and freeze the other weights.\n",
    "This is because I am unfortunately using CPU environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Users/yumikondo/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:1349: calling reduce_mean (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "keep_dims is deprecated, use keepdims instead\n"
     ]
    }
   ],
   "source": [
    "from keras.callbacks import EarlyStopping, ModelCheckpoint\n",
    "from keras.optimizers import SGD, Adam, RMSprop\n",
    "\n",
    "dir_log = \"logs/\"\n",
    "try:\n",
    "    os.makedirs(dir_log)\n",
    "except:\n",
    "    pass\n",
    "\n",
    "\n",
    "BATCH_SIZE   = 32\n",
    "generator_config['BATCH_SIZE'] = BATCH_SIZE\n",
    "\n",
    "early_stop = EarlyStopping(monitor='loss', \n",
    "                           min_delta=0.001, \n",
    "                           patience=3, \n",
    "                           mode='min', \n",
    "                           verbose=1)\n",
    "\n",
    "checkpoint = ModelCheckpoint('weights_yolo_on_voc2012.h5', \n",
    "                             monitor='loss', \n",
    "                             verbose=1, \n",
    "                             save_best_only=True, \n",
    "                             mode='min', \n",
    "                             period=1)\n",
    "\n",
    "\n",
    "optimizer = Adam(lr=0.5e-4, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0)\n",
    "#optimizer = SGD(lr=1e-4, decay=0.0005, momentum=0.9)\n",
    "#optimizer = RMSprop(lr=1e-4, rho=0.9, epsilon=1e-08, decay=0.0)\n",
    "\n",
    "model.compile(loss=custom_loss, optimizer=optimizer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Users/yumikondo/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:1349: calling reduce_mean (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "keep_dims is deprecated, use keepdims instead\n",
      "Epoch 1/50\n",
      "535/536 [============================>.] - ETA: 24s - loss: 3.4066Epoch 00001: loss improved from inf to 3.40433, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 13142s 25s/step - loss: 3.4043\n",
      "Epoch 2/50\n",
      "535/536 [============================>.] - ETA: 20s - loss: 2.2446Epoch 00002: loss improved from 3.40433 to 2.24404, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 11008s 21s/step - loss: 2.2440\n",
      "Epoch 3/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 1.8147Epoch 00003: loss improved from 2.24404 to 1.81350, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10618s 20s/step - loss: 1.8135\n",
      "Epoch 4/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 1.5694Epoch 00004: loss improved from 1.81350 to 1.56886, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10390s 19s/step - loss: 1.5689\n",
      "Epoch 5/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 1.4156Epoch 00005: loss improved from 1.56886 to 1.41503, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10414s 19s/step - loss: 1.4150\n",
      "Epoch 6/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 1.3039Epoch 00006: loss improved from 1.41503 to 1.30439, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10340s 19s/step - loss: 1.3044\n",
      "Epoch 7/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 1.2295Epoch 00007: loss improved from 1.30439 to 1.22947, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10236s 19s/step - loss: 1.2295\n",
      "Epoch 8/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 1.1702Epoch 00008: loss improved from 1.22947 to 1.16959, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10268s 19s/step - loss: 1.1696\n",
      "Epoch 9/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 1.1217Epoch 00009: loss improved from 1.16959 to 1.12201, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10243s 19s/step - loss: 1.1220\n",
      "Epoch 10/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 1.0834Epoch 00010: loss improved from 1.12201 to 1.08334, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10162s 19s/step - loss: 1.0833\n",
      "Epoch 11/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 1.0535Epoch 00011: loss improved from 1.08334 to 1.05358, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10433s 19s/step - loss: 1.0536\n",
      "Epoch 12/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 1.0263Epoch 00012: loss improved from 1.05358 to 1.02609, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10143s 19s/step - loss: 1.0261\n",
      "Epoch 13/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 1.0024Epoch 00013: loss improved from 1.02609 to 1.00246, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10600s 20s/step - loss: 1.0025\n",
      "Epoch 14/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.9837Epoch 00014: loss improved from 1.00246 to 0.98375, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10167s 19s/step - loss: 0.9837\n",
      "Epoch 15/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 0.9699Epoch 00015: loss improved from 0.98375 to 0.97007, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10219s 19s/step - loss: 0.9701\n",
      "Epoch 16/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 0.9485Epoch 00016: loss improved from 0.97007 to 0.94880, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10260s 19s/step - loss: 0.9488\n",
      "Epoch 17/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 0.9345Epoch 00017: loss improved from 0.94880 to 0.93443, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10259s 19s/step - loss: 0.9344\n",
      "Epoch 18/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 0.9226Epoch 00018: loss improved from 0.93443 to 0.92268, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10229s 19s/step - loss: 0.9227\n",
      "Epoch 19/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 0.9147Epoch 00019: loss improved from 0.92268 to 0.91463, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10367s 19s/step - loss: 0.9146\n",
      "Epoch 20/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.9043Epoch 00020: loss improved from 0.91463 to 0.90443, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10166s 19s/step - loss: 0.9044\n",
      "Epoch 21/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.8929Epoch 00021: loss improved from 0.90443 to 0.89287, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10132s 19s/step - loss: 0.8929\n",
      "Epoch 22/50\n",
      "535/536 [============================>.] - ETA: 20s - loss: 0.8905Epoch 00022: loss improved from 0.89287 to 0.89053, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 11085s 21s/step - loss: 0.8905\n",
      "Epoch 23/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.8810Epoch 00023: loss improved from 0.89053 to 0.88130, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10108s 19s/step - loss: 0.8813\n",
      "Epoch 24/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.8735Epoch 00024: loss improved from 0.88130 to 0.87398, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10076s 19s/step - loss: 0.8740\n",
      "Epoch 25/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.8667Epoch 00025: loss improved from 0.87398 to 0.86716, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10111s 19s/step - loss: 0.8672\n",
      "Epoch 26/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.8581Epoch 00026: loss improved from 0.86716 to 0.85846, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10102s 19s/step - loss: 0.8585\n",
      "Epoch 27/50\n",
      "535/536 [============================>.] - ETA: 21s - loss: 0.8519Epoch 00027: loss improved from 0.85846 to 0.85145, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 11597s 22s/step - loss: 0.8515\n",
      "Epoch 28/50\n",
      " 70/536 [==>...........................] - ETA: 2:45:17 - loss: 0.8663"
     ]
    }
   ],
   "source": [
    "model.fit_generator(generator        = train_batch_generator, \n",
    "                    steps_per_epoch  = len(train_batch_generator), \n",
    "                    epochs           = 50, \n",
    "                    verbose          = 1,\n",
    "                    #validation_data  = valid_batch,\n",
    "                    #validation_steps = len(valid_batch),\n",
    "                    callbacks        = [early_stop, checkpoint], \n",
    "                    max_queue_size   = 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[FairyOnIce/ObjectDetectionYolo](https://github.com/FairyOnIce/ObjectDetectionYolo)\n",
    " contains this ipython notebook and all the functions that I defined in this notebook. \n",
    "\n",
    "By accident, I stopped a notebook.\n",
    "Here, let's resume the training.."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.8487Epoch 00001: loss improved from inf to 0.84864, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10093s 19s/step - loss: 0.8486\n",
      "Epoch 2/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.8407Epoch 00002: loss improved from 0.84864 to 0.84071, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 9974s 19s/step - loss: 0.8407\n",
      "Epoch 3/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 0.8348Epoch 00003: loss improved from 0.84071 to 0.83478, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10325s 19s/step - loss: 0.8348\n",
      "Epoch 4/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 0.8326Epoch 00004: loss improved from 0.83478 to 0.83307, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10211s 19s/step - loss: 0.8331\n",
      "Epoch 5/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 0.8289Epoch 00005: loss improved from 0.83307 to 0.82875, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10407s 19s/step - loss: 0.8287\n",
      "Epoch 6/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.8233Epoch 00006: loss improved from 0.82875 to 0.82336, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10118s 19s/step - loss: 0.8234\n",
      "Epoch 7/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.8226Epoch 00007: loss improved from 0.82336 to 0.82245, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10116s 19s/step - loss: 0.8225\n",
      "Epoch 8/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.8165Epoch 00008: loss improved from 0.82245 to 0.81658, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10114s 19s/step - loss: 0.8166\n",
      "Epoch 9/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 0.8120Epoch 00009: loss improved from 0.81658 to 0.81197, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10435s 19s/step - loss: 0.8120\n",
      "Epoch 10/50\n",
      "535/536 [============================>.] - ETA: 18s - loss: 0.8085Epoch 00010: loss improved from 0.81197 to 0.80853, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10108s 19s/step - loss: 0.8085\n",
      "Epoch 11/50\n",
      "535/536 [============================>.] - ETA: 29s - loss: 0.8038 Epoch 00011: loss improved from 0.80853 to 0.80450, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 16065s 30s/step - loss: 0.8045\n",
      "Epoch 12/50\n",
      "535/536 [============================>.] - ETA: 19s - loss: 0.8010Epoch 00012: loss improved from 0.80450 to 0.80065, saving model to weights_yolo_on_voc2012.h5\n",
      "536/536 [==============================] - 10262s 19s/step - loss: 0.8006\n",
      "Epoch 13/50\n",
      "106/536 [====>.........................] - ETA: 2:27:29 - loss: 0.7866"
     ]
    }
   ],
   "source": [
    "model.load_weights('weights_yolo_on_voc2012.h5')\n",
    "model.fit_generator(generator        = train_batch_generator, \n",
    "                    steps_per_epoch  = len(train_batch_generator), \n",
    "                    epochs           = 50, \n",
    "                    verbose          = 1,\n",
    "                    #validation_data  = valid_batch,\n",
    "                    #validation_steps = len(valid_batch),\n",
    "                    callbacks        = [early_stop, checkpoint], \n",
    "                    max_queue_size   = 3)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
